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ABSTRACT: 

Gesture Controlled Car is a robot which can be controlled by simple human gestures. The user just needs to 
wear a gesture device in which a sensor is included. The sensor will record the movement of hand in a 
specific direction which will result in the motion of the robot in the respective directions. The robot and the 
Gesture instrument are connected wirelessly through radio waves. User can interact with the robot in a more 
friendly way due to the wireless communication. We can control the car using accelerometer sensors 
connected to a hand glove. The sensors are intended to replace the remote control that is generally used to 
run the car. It will allow user to control the forward, backward, leftward and rightward movements, while 
using the same accelerometer sensor to control the throttle of the car. Movement of car is controlled by the 
differential mechanism. The mechanism involves the rotation of both forth & rear wheels of left or right side 
to move in the anticlockwise direction and the other pair to rotate in the clockwise direction which makes 
the car to rotate about its own axis without any kind of forward or backward motion. The main advantage of 
this mechanism is the car with this mechanism can take sharp turn without any difficulty. The design and 
implementation of a gesture control robotic arm using flex sensor is proposed. The robotic arm is designed 
in such a way that it consists of four movable fingers, each with three linkages, an opposing thumb, a 
rotating wrist and an elbow. The robotic arm is made to imitate the human hand movements using a hand 
glove. 

Keywords: Robotic Arm, Flex Sensor, Wireless Module, Accelerometer 


1. INTRODUCTION 

Nowadays, robotics are becoming one of the most advanced in the field of technology. A Robot is an 
electro-mechanical system that is operated by a computer program. Robots can be autonomous or semi- 
autonomous. An autonomous robot is not controlled by human and acts on its own decision by sensing its 
environment. Majority of the industrial robots are autonomous as they are required to operate at high speed 
and with great accuracy. But some applications require semi-autonomous or human controlled robots. Some 
of the most commonly used control systems are voice recognition, tactile or touch controlled and motion 
controlled. A Gesture Controlled robot is a kind of robot which can be controlled by your hand gestures not 
by old buttons. You just need to wear a small transmitting device in your hand which included an 
acceleration meter. This will transmit an appropriate command to the robot so that it can do whatever we 
want. The transmitting device included a ADC for analog to digital conversion and an encoder IC(HT12E) 
which is use to encode the four bit data and then it will transmit by an RF Transmitter module. At the 
receiving end an RF Receiver module receives the encoded data and decodes it by and decoder IC (HT12D). 
This data is then processed by a microcontroller and finally our motor driver to control the motors. Now its 
time to break the task in different module's to make the task easy and simple any project become easy or 
error free if it is done in different modules. As our project is already divided into two different part 
transmitter and receiver. The applications of robotics mainly involve in automobiles, medical, construction, 
defense and also used as a fire fighting robot to help the people from the fire accident. But, controlling the 
robot with a remote or a switch is quite complicated. So, a new project is developed that is, an accelerometer 
based gesture control robot. The main goal of this project is to control the movement of the robot with hand 
gesture using accelerometer. The robot is usually an electro-mechanical machine that can perform tasks 
automatically. Some robots require some degree of guidance, which may be done using a remote control or 
with a computer interface. Robots can be autonomous, semi-autonomous or remotely controlled. Robots 
have evolved so much and are capable of mimicking humans that they seem to have a mind of their own. 
1.1Problem statement 

The traditional wired buttons controlled robot becomes very bulgy and it also limits the distance the robot 
goes. The Wireless Hand controlled Robot will function by a wearable hand glove from which the 
movements of the hand can be used as the input for the movement of the robot. The basic idea of our project 
is to develop a system (Robot) which can recognize the Human Interaction with it to accomplish the certain 
tasks assigned to it. In our project we will design a wearable Hand Glove which will contain the sensors 
mounted on it to capture the movement of the hand and convert the raw mechanical data into electrical form. 
This data will be further processed and converted into an understandable format for the lily pad mounted on 
the Glove. This lily pad will act as a transmitter of the data for wireless communication purpose. Once the 
transmitted data is received by the receiver module which will be connected to the Microcontroller, it will be 
processed and further sent to the Microcontroller. Microcontroller will deduce the commands and 
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accordingly it will actuate the motor drivers to control the Motors for various tasks on the robot. 

1.2. Objective 

The aim of the project is to develop a human machine interface used for control robot arm. Our objective 1s 
to make this device simple as well as cheap so it can be produced and used for number of purposes. The 
objective of this project is to build a car that can be controlled by gesture wirelessly. In this project user is 
also able to control motions of the car by wearing controller glove and performing predefined gestures. This 
can be also used in many potential applications such as wireless controller car racing etc. 


2s Scope 

° Wireless controlled robots are very useful in many applications like remote surveillance, military 
etc. 

. Hand gesture controlled robot can be used by physically challenged in wheelchairs. 

° Hand gesture controlled industrial grade robotic arms can be developed. 

° Entertainment applications — Most videogames today are played either on game consoles, arcade 


units or PCs, and all require a combination of input devices. Gesture recognition can be used to truly 
immerse a players in the game world like never before. 

. Automation systems — In homes, offices, transport vehicles and more, gesture recognition can be 
incorporated to greatly increase usability and reduce the resources necessary to create primary or secondary 
input systems like remote controls, car entertainment systems with buttons or similar. 


° An easier life for the disabled — One of the biggest challenges faced today is providing separate and 
equally non cumbersome services to the differently abled and handicapped. While there are special 
provisions around the world, there’s still huge room for improvement to bring all lives on equal footing. 
Gesture recognition technology can eliminate a lot of manual labor and make life much easier for those who 
aren’t as fortunate as most of us are. 


These are just a handful of the places and situations in which gesture recognition technology can be 
implemented, and as is evident, can totally change the way we interact with the world around us, not only at 
home, but in commercial venues as well. In fact, a South African company had come up with an innovative 
machine placed at the Tambo International Airport that detected travelers who yawned or looked sleepy and 
dispensed free cups of coffee. Although it used only basic facial and gesture recognition technology, it is 
nonetheless an interesting look into what can be done with this technology. Currently, there aren’t too many 
gesture recognition applications available for public use, but interestingly, despite its potential for real world 
applications, gesture recognition technology is actually dominated by the videogame industry. Electronics 
giants Microsoft and Sony, makers of the Xbox and PlayStation line of consoles respectively, have 
incorporated gesture recognition to an extent into their entertainment systems, via extra hardware. Called 
‘Kinect’ in the case of Microsoft and the ‘PlayStation Eye/Camera’ in the case of Sony, these amazing 
devices bring us one step closer to the future. While Microsoft in 2014 has gone ahead and included the 
Kinect 2.0 camera with the Xbox One, their latest gaming console and made gesture and voice control an 
integral part of it, Sony has left the PlayStation Camera as an accessory for the PlayStation 4, instead 
focusing on traditional input methods. 


So far you came to know about Hand Gesture Controlled Robot that completely moves according to 
moments of your hand (sign of input to the device). 


1.3. Methodology 

Methodology for communication signal Transmitter Module 

An RF transmitter module is a small PCB 1.e., printed circuit board sub-assembly capable of transmitting a 
radio wave and modulating that wave to carry data. Transmitter modules are usually implemented alongside 
a micro controller which will provide data to the module which is transmitted. RF transmitters are usually 
subject to regulatory requirements which dictate the maximum allowable transmitter power output, 
harmonics and band edge requirement. 

Receiver modules 

An RF Receiver module RF433-RX is 433 MHz radio receiver receives the modulated RF signal, and then it 
demodulates. There are two types of RF receiver module. Super-regenerative modules are usually of low 
cost and low power designs using a series of amplifiers use to extract modulated data from a carrier wave. 
Super-regenerative modules are generally imprecise as their frequency of operation varies in a fair amount 
with temperature and power supply voltage. Super heterodyne receivers having a performance advantage 
over super-regenerative; they offer increased an accuracy and stability over a large voltage and temperature 
range. This stability comes from a fixed crystal design which in turn leads to a comparatively more 
expensive product. Radio receiver which receives the transmitted coded from the remote place these codes 
are converted to digital format and output is available to the pin 


no 2 of the ic2 master microcontroller; this is the pin of inbuilt art of the microcontroller. We Based on the 
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input codes master will give command to slave microcontroller and robot will behave as follows. 


° Moves in forward direction 

° Moves in reverse direction, 

° Speed controls in both the direction 

. It can even turn left or right while moving forward or in reverse direction. 

° In case of bump, moves reverse turn left or right and wail for the next instruction. 
° On the spot left or right turn to pass through the narrow space 

° We have also added head light, back light and turning lights to left a right. 


Methodology for Motion Control 

L293D is a dual H-bridge motor driver integrated circuit (IC). Motor drivers act as current amplifiers as they 
take a low-current control signal and provide a higher-current signal. This higher current signal is used to 
drive the motors. L293D contains two inbuilt H-bridge driver circuits. In common mode of operation, two 
DC motors can be driven simultaneously, both in forward and reverse direction. The motor operations of 
two motors can be controlled by input logic. When an enable input is high, the associated driver gets 
enabled. As a result, the outputs become active and work in phase with their inputs. Similarly, when the 
enable input is low, that driver is disabled, and their outputs are off and in the high-impedance state. This 
project controls a remote robot through RF. The ordinary 433 MHz RF modules are used in this project. 
AT89C51 microcontroller is used in this project. 

This robot can perform their operations without direct human guidance. They are used basically for 
industrial applications and can be made laser guided. Navigation is achieved by one of the several means, 
including following a path defined by buried inductive wires, surface mounted magnetic or optical strips; or 
alternatively by the way of laser guidance. This is an improved version of my previous robot which we 
designed years ago. Intelligent spy robot project has been designed for the spying purpose .it is radio 
controlled and can be operated at a radial distance of 100m radius. Most probably our army youth need to 
venture into the enemy area just to track their activities. 

Which is often a very risky job and may cost precious life? Such dangerous job could be done using small 
spy robot all the developed and advance nations are in the process of making it, a robot that can fight against 
enemy. Our robot us just a step towards similar activity. 

This robot is radio operated which is, self-powered, and has all the controls like a normal car. A laser gun 
has been installed on it so that it can fire on enemy remotely whenever required; this is not possible until a 
wireless camera is installed. Wireless camera will send real time video and audio signals which could be 
seen on a remote monitor and 224 actions can be taken accordingly. Being in size small of it, will not be 
tracked by enemy on his radar. Robot silently enter into enemy canopy or tent and send us all the 
information through its’ tiny camera eyes. It can also be used for suicide attack, if required. Heart of our 
robot is microcontroller 8051 family, we are using at89C51 in two microcontrollers where first 
microcontroller which acts as master controller, decodes all the commands received from the transmitter and 
give commands to slave microcontroller. Slave microcontroller is responsible for executing all the 
commands received from the master and also generating pulse width modulation pulses for the speed control 
driver circuit which drives 4 nos. of motors. Two no bumper switch is added bmp 1 and bmp? so that in case 
of accident our battery dose not drains out. 

Both the motors will stop instantly and after few second robots will move in opposite direction take turn to 
left or right direction and stops and stop. Navigation and Dead Reckoning, Tilt Compensation in inertial 
sensors, 3D-Gaming. Transmitter receives serial data and transmits it wirelessly through RF through its 
antenna connected at pin4. The transmission occurs at the rate of 1Kbps- 1O0Kbps. The transmitted data is 
received by an RF receiver operating at the same frequency as that of the transmitted. 

Transmission through RF (Radio frequency) is better than IR (infrared) because of many reasons. Firstly, 
signals through RF can travel through larger distances making it suitable for long range applications. Also, 
while IR mostly operates in line Of sight mode, RF signals can travel even when there is an obstruction 
between transmitter & receiver. Next, RF transmission is more strong and reliable than IR transmission. RF 
communication uses a specific frequency unlike IR signals which are affected by other IR emitting sources. 
This RF module comprises of an RF Transmitter and an RF Receiver. The transmitter/receiver (TX/RX) pair 
operates at a frequency of 433MHz an RF transmitter receives serial data and transmits it wirelessly through 
RF through its antenna connected at pin4. The transmission occurs at the rate of 1Kbps-10Kbps. The 
transmitted data is received by an RF receiver operating at the same frequency as that of the transmitter. 
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Fig.1.0 Movement Signs 
1. LITERATURE REVIEW 


Moniruzzaman Bhuiyan and Rich Picking in Centre for Applied Internet Research (CAIR), 
Glyndwr University, and Wrexham, UK, proposed a review of the history of Gesture controlled user 
interface (GCUI), and identifies trends in technology, application and usability. Their findings conclude 
that GCUI [1] affords realistic opportunities for specific application areas, and especially for users who are 
uncomfortable with more commonly used input devices. They have tried collated chronographic research 
information which covers the past 30years. They investigated different types of gestures, its users, 
applications, technology, issues addressed, results and interfaces from existing research. They consider the 
next direction of gesture controlled user interfaces as rich user interface using gestures seems appropriate 
for current and future ubiquitous and ambient devices. Moniruzzaman Bhuiyan, Rich Picking of Institute 
of Information technology, University of Dhaka, Dhaka, Bangladesh; Centre for Applied Internet 
Research, Glyndwr University, Wrexham, United Kingdom on September 2011 in Journal of Software 
Engineering and to meet the challenges of ubiquitous computing, ambient technologies and an increasingly 
older population, researchers have been trying to break away from traditional modes of interaction. A 
history of studies over the past 30 years reported in this paper suggests that Gesture Controlled User 
Interfaces (GCUI) now provide realistic and affordable opportunities, which may be appropriate for older 
and disabled people. They have developed a GCUI prototype application, called Open Gesture, to help users 
carry out everyday activities such as making phone calls, controlling their television and performing 
mathematical calculations. Open Gesture uses simple hand gestures to perform a diverse range of tasks via 
a television interface. They describe Open Gesture and report its usability evaluation. They conclude that 
this inclusive technology offers some potential to improve the independence and quality of life of older and 
disabled users along with general users, although there remain significant challenges to be overcome. 
Stefan Waldherr, Roseli Romero, Sebastian Thrun describes a gesture interface for the control of a mobile 
robot equipped with a manipulator. The interface uses a camera to track a person and recognize gestures 
involving arm motion. A fast, adaptive tracking algorithm enables the robot to track and follow a person 
reliably through office environments with changing lighting conditions. Two alternative methods for 
gesture recognition are compared: a template based approach and a neural network approach. Both are 
combined with the Viterbi algorithm for the recognition of gestures defined through arm motion (in 
addition to static arm poses). Results are reported in the context of an interactive clean-up task, where a 
person guides the robot to specific locations that need to be cleaned and instructs the robot to pick up trash. 


2. SIMULATION WORK 





3.1. RF pair 


Volume 2, Issue 5, 20181225 


e-ISSN : 2620 3502 


ji’ ; 
ey International Journal on Integrated Education p-ISSN : 2615 3785 


i 


Fig.2.0 Picture of Object 


A gesture controlled robot is controlled by using hand in place of any other method like buttons or 
joystick. Here one only needs to move hand to control the robot. A transmitting device is used in your 
hand which contains RF Transmitter and accelerometer. This will transmit command to robot so that it 
can do the required task like moving forward, reverse, turning left, turning right and stop. All these 
tasks will be performed by using hand gesture. 


Here the most important component is accelerometer. Accelerometer is a 3 axis acceleration 
measurement device with +-3g range. This device is made by using polysilicon surface sensor and 
signal conditioning circuit to measure acceleration. The outputof this device is Analog in nature and 
proportional to the acceleration. This device measures the static acceleration of gravity when we tilt it. 
And gives a result in form of motion or vibration. 


According to the datasheet of adx1335 polysilicon surface-micro machined structure placed on 
top of silicon wafer. Polysilicon springs suspend the structure over the surface of the wafer and provide a 
resistance against acceleration forces. Deflection of the structure is measured using a differential capacitor 
which incorporates independent fixed plates and plates attached to the moving mass. The fixed plates are 
driven by 180° out-of-phase square waves. Acceleration deflects the moving mass and unbalances the 
differential capacitor resulting in a sensor output whose amplitude is proportional to acceleration. Phase- 
sensitive demodulation techniques are then used to determine the magnitude and direction of acceleration. 


Fig.3.0 accelerometer 
Pin Description of accelerometer 
1. VCC 5 volt supply should connect at this pin. 
X-OUT This pin gives an Analog output in x direction 
Y-OUT This pin give an Analog Output in y direction 
Z-OUT This pin gives an Analog Output in z direction 
GND Ground 


se es oF UU»! 


ST This pin used for set sensitivity of sensor 


3.2.Circuit Diagram and Explanation 


Gesture Controlled Robot is divided into two sections: 


1. Transmitter part 


CLD teeter tterees 


2. Receiver part 


In transmitter part an accelerometer and a RF transmitter unit is used. As we have already 
discussed that accelerometer gives an analog output so here we need to convert this analog data in to 
digital. For this purpose we have used 4 channel comparator circuits in place of any ADC. By setting 
reference voltage we gets a digital signal and then apply this signal to HT12E encoder to encode data or 
converting it into serial form and then send this data by using RF transmitter into the environment. At the 
receiver end we have used RF receiver to receive data and then applied to HT 12D decoder. This decoder 
IC converts received serial data to parallel and then read by using Arduino. According to received data we 
drive robot by using two DC motor in forward, reverse, left, right and stop direction. 
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3.3. Working of Model 


Gesture controlled robot moves according to hand movement as we place transmitter in our 
hand. When we tilt hand in front side, robot start to moving forward and continues moving forward until 
next command is given. When we tilt hand in backward side, robot change its state and start moving in 
backwards direction until other command is given. When we tilt it in left side Robot get turn left till next 
command. When we tilt hand in right side robot turned to right. And for stopping robot we keeps hand in 
stable. 





Fig.4.0 Transmitter 
3.4. Receiver circuit. 
a 
| ae 


Fig. 5.0 Receiver Circuit 


3. This transmitted signal is received by the RF receiver, demodulated and then passed onto the decoder IC. 
The decoder IC decodes the coded waveform and the original data bits are recovered. The input is a serial 
coded modulated waveform while the output is parallel. The pin 17 of the decoder IC is the Valid 
Transmission (VT) pin. A led can be connected to this pin which will indicate the status of the transmission. 
In the case of a successful transmission, the led will blink. The parallel data from the encoder is fed to the 
port lof the microcontroller. This data is in the form of bits. The microcontroller reads these bits and takes 
decisions on the basis of these bits. What the microcontroller does is, it compares the input bits with the 
coded bits which are burnt into the program memory of the microcontroller and outputs on the basis of these 
bits. Port 2 of the microcontroller is used as the output port. Output bits from this port are forwarded to the 
motor driver IC which drives the motors in a special configuration .Gesture controlled robot works on the 
principle of accelerometer which records hand movements and sends that data to the comparator which 
assigns proper voltage levels to the recorded movements. That information is then transferred to an encoder 
which makes it ready for RF transmission. On the receiving end, the information is received wirelessly via 
RF, decoded and then passed onto the microcontroller which takes various decisions based on the received 
information. These decisions are passed to the motor driver IC which triggers the motors in different 
configurations to make the robot move in a specific direction. Task was divided into two parts to make the 
task easy and simple and to avoid complexity and make it error free. The first is the CONCLUSION 


The purpose of project is to control a toy car using accelerometer sensors attached to a hand 
glove. The sensors are intended to replace the remote control that is generally used to run the car. It will 
allow us to control the forward and backward, and left and right movements, while using the same 
accelerometer sensor to control the throttle of the car. Based on the hand movements. By using the above 
mentioned components the hardware was setup, thus resulting in the formation of a robot. In order to 
implement the experiment a Dell laptop was used, whose web camera acted as the input device for 
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capturing the video. The software part was developed in Java for image processing wherein the hand 





gestures were analyzed to extract the actual direction. Eclipse Ide was used for developing the java code. 
The direction thus identified was send as characters to the robot with the help of ZigBee. XBee S2 version 
of ZigBee was used for enabling the communication. The final movement of the robot can be concluded as 
follows: At the beginning the robot was in a stop mode. As the hand moved from bottom to top, the robot 
moved in the forward direction. As the hand moved from top to bottom, the robot moved in the backward 
direction. As the hand was shown as an acute angle towards the left, the robot moved towards the left 
direction. As the hand was shown as an acute angle towards the right, the robot moved towards the right 
direction. As the hand is kept stationary with respect to the environment, the robot was in the stop mode. 
From the experiment, about 80% of the implementation worked according; the remaining was less due to 
background interference which is a negative marking to the implementation. Hand Gesture Controlled 
Robot System gives a more natural way of controlling devices. The command for the robot to navigate in 
specific direction in the environment is based on technique of hand gestures provided by the user. Without 
using any external hardware support for gesture input unlike specified existing system, user can control a 
robot from his software stationtransmitting section which includes the following components: 


Accelerometer, encoder IC, RF Transmitter Module. The second is the receiving end which 
comprises of following main components: RF Receiver Module, Decoder IC, Microcontroller, Motor 
Driver IC, and DC Motors. The accelerometer records the hand 


movements in the X and Y directions only and outputs constant analog voltage levels. These voltages 
are fed to the microcontroller which processes the input and encodes the data into digital form which is 
suitable to be transmitted through the XBee serial transmitter. Circuit for this hand gesture controlled 
robot is quite simple. As shown in above schematic diagrams, a RF pair is used for communication and 
connected with Arduino. Motor driver is connected to Arduino to run the robot. Motor driver’s input 
pin 2, 7, 10 and 15 is connected to Arduino digital pin number 6, 5, 4 and 3 respectively. Here we have 
used two DC motors to drive robot in which one motor is connected at output pin of motor driver 3 and 
6 and another motor is connected at 11 and 14. A 9 volt Battery is also used to power the motor driver 
for driving motors 


FUTURE SCOPE 


The on board batteries occupy a lot of space and are also quite heavy. We can either use some alternate 
power source for the batteries or replace the current DC Motors with ones which require less power. 


The proposed system is applicable in hazardous environment where a camera can be attached to the robot 
and can be viewed by the user who is in his station. This system can also be employed in medical field 
where miniature robot are created that can help doctors for efficient surgery operations For more efficient 
response, threshold values can be used to detect gesture and advanced features such as finger counts that 
provide different functional commands can be used. 


Entertainment applications — Most videogames today are played either on game consoles, arcade units or 
PCs, and all require a combination of input devices. Gesture recognition can be used to truly immerse a 
player in the game world like never before. 


Automation systems — In homes, offices, transport vehicles and more, gesture recognition can be 
incorporated to greatly increase usability and reduce the resources necessary to create primary or 
secondary input systems like remote controls, car entertainment systems with buttons or similar. 
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